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Summary 


this  paper  qp  extend^ the  fundamental  Identity  of  Wald 
In  the  theory  of  sequential  analysis  to  the  case  where  the 
variables  are  Markovian  rather  than  Independent. 
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ON  A  GENERALIZATION  OP  THE  FUNDAMENTAL  IDENTITY  OP  WALD 

By 

Richard  Bellman 

The  RAND  Corporation,  Santa  Monica,  California 


§1 .  Introduction 

Let  Zj  be  a  eequence  of  Independent  random  variables 
with  identical  distribution  functions,  and  let  SN  denote  tne 
partial  sum  zi  +  z2  +  *  *  *  4  zn*  *n  man>r  Interesting  situations, 
we  encounter  stochastic  processes  in  which  a  "stop  rule" 
exists,  making  N  itself  a  random  variable  which  we  write 
as  n.  Examples  of  this  occur  in  the  study  of  random  walk  in 
the  presence  of  absorbing  barriers,  in  "excluded  volume" 
problems,  in  sequential  analysis,  in  branching  processes,  and 
in  many  other  types  of  processes.  The  problem  of  determining 
the  behavior  of  the  random  variable  is,  in  many  of  these  cases, 
a  matter  of  difficulty,  and  in  other  cases,  as  yet  unresolved. 

A  technique  much  more  powerful  in  a  number  of  applications 
than  the  traditional  technique  based  upon  recurrence  relations 
was  developed  by  Wald  in  19^,  or  earlier,^1-1.  Under  light 
restrictions  upon  the  variables  z^  and  the  nature  of  the 
stop  rule,  he  showed  that 

E(<*(t)_n«Snt)  -  1  U-l) 

for  t  >  0,  where  <Ht)  le  the  characteristic  function  of  the 
random  variable  z^.  This  was  called  by  Wald  the  "fundamental 
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ldentlty",  and  la  olosely  connected  with  martingale  theory, 


The  extension  to  the  multl-dlmenelonal  case,  where  the 

z^  are  Independent  random  k-dlmenslonal  vectors  was  given  by 

Blackwell  and  Olrshlck^,  together  with  a  discussion  of  the 

determination  of  S  In  certain  cases. 

n 

In  this  paper  we  wish  to  Indicate  how  analogues  of  the 
Wald  Identity  may  be  obtained  for  the  case  where  the  z^  are 
no  longer  Independent,  but  possess  certain  Markovian  properties. 
In  the  sense  that  the  distribution  function  of  the  z^^ 
depends  only  upon  the  value  of  z^ .  More  generally,  results 
can  be  obtained  for  processes  In  which  z^+i  depends  upon  z^, 
zk-1,  •••,  for  fixed  R,  or  upon  S^.  Furthermore,  the 

results  obtained  for  the  commutative  scalar  case  can  be  extended 

to  the  non— commutative  case  where  we  have  matrix  random 

f4] 

variables L  . 

We  are  primarily  Interested  In  Indicating  the  nature  of 
the  extension  and  postpone  any  rigorous  discussion  and  treatment 
of  various  special  cases  until  a  later  date. 


§2 .  Markovian  Case 

Let  dQ{z,x)  be  the  distribution  function  of  z  -  zk+i# 
given  that  z.  -  x.  Then  4i(x,  t)  -  h(e^1)  -  j  CDetzdQ(z,  x), 

*  -CD 

given  Zq  -  x,  and 

tsN 

4nU,  t)  -  E(e  |i0  •  x)  - 

J*  00»tZ*M-i(z’  OdOU,  *)>  N  >  2> 

—00 


(2-1) 
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Undtr  various  assumptions  concerning  dG(z,  x),  e.g.,  a  finite 
range  for  z,  we  have,  asymptotically  as  N  — >  co  , 

<♦„(*,  t)  S  A(t)H  V(x,  t).  (2.2) 

Note  that  A(0)  •  1,  and  that  Y(x,  0)  is  Independent  of  x. 
Following  the  Wald  proof  of  (l.l),  as  In  ^ ,  we  have 

tsn  ♦  t(SN  -  S  ) 

<^N(x,  t)  -  PNE(e  I  zQ  -  x)  +  rN,(2.3) 

where  is  the  probability  that  n  £  N,  and  n  Is,  aa  above,  the 
stochastic  variable  determined  by  the  stop  rule. 

Hence 

t  S 

PNU,  t)  -  PNE(e  t)|zQ  -  x)  +  rN  (2.4) 

Letting  N  — )  oo  ,  we  obtain 
t  S 

E(e  n  11m  ^N-n^n'  fc)  /  ^NU,  ^  I  z0  “  x)  "  1 

N  "^  °°  (2.5) 

Using  (2.2),  we  aee  that  the  analogue  of  Wald'a  Identity  la 

E(e  nv(zn,  t)  /  >(t)ny(x,  t)  |  zQ  -  x)  -  1.  (2.6) 

Still  proceeding  formally,  let  us  differentiate  (2.6) 

with  respect  to  t,  and  set  t  -  0.  A  rigorous  discussion  of 

i  S I 

differentiation  of  Wald’s  Identity  Is  given  by  Wald  In 
The  result  Is 


£(Sn)  -  A'  (0 ) E (n )  +  *(x)  -  E(<* (zn) ) , 


(2-7) 
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where  ^(x)  -  lY/ltl 

1 1-0 

In  order  to  use  this  to  compute  E(n),  If  X '  (0)  +  0,  we  must 

have  some  Information  concerning  the  values  of  z  and  3  at 

n  n 

the  stage  where  the  stop  rule  Is  enforced.  In  many  cases,  e.g., 
random  walk  with  absorbing  barriers,  this  Information  is  readily 
derivable  from  the  process  itself. 


§3*  Other  Types  of  Identities 

The  advantage  of  using  the  exponential  function  resides 

In  Its  simple  functional  equation.  In  place  of  considering  the 
«  t  N 

function  e  ,  consider  T  (z„  +  c),  where  c  Is  a  constant. 

k-1  * 

Using  the  same  arguments  as  above,  we  have 


E(TT  Uk  +  c)  /  (a  ♦  c)n)  -  1, 
k-1  K 


where  a  -  E(zk),  for  the  case  where  the  z, are  independent 
variables,  and  |a  +  c|  >1. 


§4.  Matrix  Variables 

Let  us  also  point  out  that  analogous  results  may  be 
obtained  for  the  non-coramutative  case  where  we  have  matrix 
random  variables,  cf.  [4]  for  a  discussion  of  some  problems 
In  this  field. 

There  are  several  possible  generalizations  of  the  scalar 


case.  We  may  consider 


1-23-56 

P-777 

— 5— 


K([f  (zk  ♦  C)](A  4-  c)-«)  , 
k-1  K 


(1) 


or 


KCT  IZv  +  c|]|a  ♦  c|^)  , 

k-l  K 


(2) 


where  |  |  represents  the  determinant  of  the  random  matrix 

variable  +  C,  with  C  a  fixed  matrix. 

In  turn,  we  may  consider 


tr(T(21+Z2+---+Zn) 


♦  (T)-")  . 


(3) 


wh.re  ♦  (T)  -  ,C.*rCTZ), _ 

Formulas  corresponding  to  those  derived  in  the  one- 
dimensional  case  may  be  obtained  for  these. 
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